Colouring the Near-Infrared

Current digital camera sensors are inherently sensitive to the near-infrared part of the spectrum. To prevent the near-IR contamination of images, an IR blocking filter (hot mirror) is placed in front of the sensor. In this work, we start by replacing the camera’s hot mirror by a piece of clear glass, thus making the camera sensitive to both visible and near-IR light. Using a pair of lens-mounted filters, we explore the differences in operating the camera to take visible and near-IR images of a given scene. Our aim is to enhance the visible images using near-IR information. To do so, we first discuss the physical causes of differences between visible and near-IR natural images, and remark that these causes are not correlated with a particular colour, but with atmospheric conditions and surface characteristics. We then investigate image enhancement by considering the near-IR channel as either colour, luminance, or frequency counterpart to the visible image and conclude that using information from two different colour encodings, depending on the image content, produces vivid, contrasted images that are pleasing to the observers. Introduction In contrast to film photography, digital sensors are very sensitive not only to the visible spectrum of 400-700 nm, but also to the near-infrared (IR) part of the electromagnetic spectrum: 7001100 nm. In fact, silicon is sensitive to near-IR that an infrared filter, “hot mirror”, is placed in front of most digital camera sensors to prevent near-IR contamination of the colour signals. Effectively, this means that by removing the hot mirror, most consumer cameras automatically become near-IR capable. Indeed, this is being routinely done by amateur photographers for specific applications, such as astrophotography or landscape photography. Additionally, while near-IR film existed, its response was very slow, whereas digital cameras have similar exposure time in the visible or the near IR, thus allowing a greater diversity of images. Our goal here is not to restrict ourselves to near-IR images, but rather to use information from both the visible and near-IR parts of the spectrum to enhance digital images. To do so requires replacing the hot mirror of a camera by an equivalent (in size) piece of clear glass. Using an IR-block or IR-pass filter in front of the lens allows us to capture either a near-IR or visible image of any given scene. Despite the continued presence of the colour filter array in front of the sensor, near-IR images are effectively grayscale images if white balanced appropriately. It implies that one can consider the combination of visible and near-IR information as a four-channel image that covers a spectrum range of 400-1100 nm. To represent that information in a meaningful manner, one must have a specific application in mind. Ours is to enhance visible images using near-IR inherent properties, yielding images with high contrast, “dramatic” lighting, and more vivid colours. To do so, we do not consider the near-IR channel as a “fourth colour”, but rather as containing spatial information and lightness. We investigate different manners to combine that information with conventional RGB images and perform a psychophysical evaluation to find out which representation observers prefer. The rest of this paper is organised as follows: in Section 2 we review the different approaches to treat near-IR data that have been proposed to date. In Section 3, we make the case for using near-IR as spatial and lightness information, and review the physical phenomena that motivate our approach. Section 4 deals with image (RGB and near-IR) acquisition parameters that allow us to obtain suitable images for further processing. The experiments and results are presented and commented in Section 5, while Section 6 concludes the paper.